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Correlated Sources over Broadcast Channels 

Paolo Minero and Young-Han Kim 



Abstract — The problem of reliable transmission of correlated 
sources over the broadcast channel, originally studied by Han 
and Costa, is revisited. An alternative characterization of their 
sufficient condition for reliable transmission is given, which 
includes results of Marton for channel coding over broadcast 
channels and of Gray and Wyner for distributed source coding. 
A "minimalistic" coding scheme is presented, which is based on 
joint typicality encoding and decoding, without requiring the use 
of Cover's superposition coding, random hashing, and common 
part between two sources. The analysis of the coding scheme is 
also conceptually simple and relies on a new multivariate covering 
lemma and an application of the Fourier-Motzkin elimination 
procedure. 



I. Introduction 

We study the problem of broadcasting two arbitrarily corre- 
lated sources over a general two-receiver discrete memoryless 
broadcast channel (DM-BC) as depicted in Figure[T] A discrete 
memoryless stationary source produces n independent copies 
{81,82) = {{8i,t, 82,t)}i^i of a pair of random variables 
{81, 82) ^ p{si, S2). The encoder maps the source sequences 
{81,82) into a sequence X" from a finite set X and broad- 
casts it to two separate receivers. The communication channel 
is memoryless and modeled by a transition probability matrix 
p{yi,y2\x), which maps each channel input x into a pair 
of finite valued output symbols (?/i,j/2)- Decoder 1 maps 
the channel output sequence into an estimate S*" of the 
source sequence 8". Similarly, decoder 2 maps KJ' into an 
estimate 5*2 of the source sequence 5*2 ■ For a given encoder 
and decoders, the average probability of error is defined as 

Pt{{S^,S^)^{S^,8^)}. 

We say that the sources (51,52) can be reliably transmitted 
over the DM-BC p{yi,y2\x) if there exists a sequence of 
encoders and decoder pairs such that the average probability 
of error vanishes as n ^ cxd. 

Han and Costa |fT6l provide a sufficient condition for 
reliable transmission of the sources {81,82) over a general 
DM-BC p{yi,y2\x)- Necessary conditions are derived by 
Gohari and Anantharam lfT4ll . and Kramer et al. ifTSl . Finding 
matching sufficient and necessary conditions is still an open 
problem in general. 

Two special classes of sources and channels have been 
studied extensively in the literature. On the one hand, suppose 
81 = {Wo,Wi) and ^2 = (H^o, ^^2), where Wq, Wi, and W2 
are three mutually independent random variables of entropies 
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Fig. 1. The two-receiver broadcast channel with arbitrarily correlated sources. 



Rq, Ri, and i?2, respectively. The (closure of the) set of triples 
{Rq, Ri, R2) which can be reliably transmitted is called the 
capacity region of the DM-BC. Inner bounds on the capacity 
region of a general DM-BC are developed in ||3], fS), ifTSll . 
mi, II2TI . II24I . and the most general known achievable rate 
region is due to Marton 1211 . which is tight for several classes 
of broadcast channels. On the other hand, suppose the DM-BC 
is noiseless with Yi = {Xo,Xi), Y2 = {Xo,X2), where Xq, 
Xi, and X2 have cardinalities 2^°, 2^^, and 2^^, respectively. 
The (closure of the) set of triples {Ro,Ri,R2) which can 
be reliably transmitted is called the distributed source coding 
region, and its complete characterization is given by Gray and 
Wyner Tl5\ . 

For transmission of arbitrarily correlated sources over a 
general noisy DM-BC, conclusive results are known for the 
case in which either source or channel has a degraded struc- 
ture. More specifically, Han and Costa's sufficient condition 
matches known necessary conditions if decoder 1 is interested 
in recovering both source sequences {8^,82) while decoder 
2 is interested in 82 only (degraded source sets) |l9l or if the 
DM-BC is degraded, or more generally, "more capable" ||6l, 
fTsl. 

In this paper, we provide an alternative characterization 
to the sufficient condition of Han and Costa for reliable 
transmission of correlated sources. This new characterization, 
while algebraically equivalent to Han and Costa's original 
sufficient condition, is simpler and includes in a straightfor- 
ward manner Marlon's inner bound on the broadcast channel 
capacity region. Gray- Wyner distributed source coding region, 
and other aforementioned known results. In particular, the new 
characterization does not involve the common part of the two 
sources in the sense of Gacs, Korner, and Witsenhausen lfT2l . 
||26l . This implies that there is no special role played by 
the common part of the sources, confirming the standard 
engineering intuition. It is interesting to observe the difference 
to the problem of reliable transmission of correlated sources 
over multiple access channels. For this problem, the common 
part of the sources plays a pivotal role in Cover, El Gamal, and 
Salehi's random coding construction [4] by inducing coherent 
transmission from separate encoders. 
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The highlight of the paper is a simple coding scheme for 
the sufficient condition and its analysis. This coding scheme 
is based on joint typicality encoding and decoding, and does 
not involve random hashing (Slepian-Wolf binning), Cover's 
superposition coding, or rate splitting as in the original coding 
scheme by Han and Costa. The performance of the coding 
scheme is analyzed via a new multivariate covering lemma and 
an application of the Fourier-Motzkin elimination procedure. 

The rest of the paper is organized as follows. In the next 
section, we review Han and Costa's sufficient condition (with 
a recent correction by Kramer and Nair ifTTl ). Section III 
presents the alternative characterization of the sufficient con- 
dition. The proof of this new coding theorem is given in 
Section IV. We conclude the paper in Section V with a 
discussion on implications of our coding scheme, and a sketch 
of yet another coding scheme for broadcasting correlated 
sources. Throughout the paper, we use the notation in [SI. 

II. Han AND Costa's Coding Theorem 

For convenience, we recall here the coding theorem of Han 
and Costa with a few minor changes. 

Theorem 1 ( H^ . ^TTi): The pair of sources {81,82) can 
be reliably transmitted over the DM-BC p{yi,y2\x) if 

H{Si) < liUo, Ui,Si;Yi) - /(C/q, t/i; ^sl^i), 
H{S2) < I{Uo,U2,S2;Y2) - I{Uo,U2;8i\S2), 
H{8i,S2) < I{K,Uo,Ui, 8i;Yi) + I{U2, 82;Y2\K,Uo) 

-I{Ui,8i;U2,82\K,Uo), (1) 
H{8uS2) < I{Ui, 8i;Y,\K, Uo) + I{K, Uo, U2, 82; I2) 

-I{Ui,8i;U2,82\K,Uo), (2) 
^2) < /(f/o, C/i, ^1; + /(C/o, U2, 82; Y2) 

- I{Ui,8i;U2, 82\K, Uo) - I {81, 82; K, Uo), 

(3) 

for some p{uo,ui,U2,x\si,S2). Here K = f{8i) = 
§{82) denotes the common variable in the sense of Gacs, 
Korner, and Witsenhausen, and the auxiliary random variable 
Uq has the cardinality bound \Uo\ < min{|A:'||5i||iS2| + 
4,|yi||3^2||5i||52|+4}. 

The coding scheme by Han and Costa requires three aux- 
iliary random variables (C/q, C^i, £^2)- Roughly speaking, Uq 
serves as a "cloud center" distinguishable by both receivers 
and carries the common part K and hash indexes of (Si, 82), 
Ui and U2 are codewords within the cloud centers which 
encode the remaining uncertainty about 81 and ^2, respec- 
tively. Finally, Marton's subcode generation technique |21 1 is 
employed to obtain arbitrary correlation among (Uo, Ui,U2)- 
For details of this interpretation, see lfT6l . 

III. An Alternative Characterization 
We have the following: 

Theorem 2: The sources (81,82) can be reliably transmit- 
ted over the DM-BC p{yi,y2\x) if 

H{8i) < I{Uo, C/i, Si; Yi) - I{Uo, Ui; S2IS1), 
H{82) < I{Uo,U2,82;Y2) - I{Uo,U2;8i\82), 



H{8i,82) < I{Uo,Ui,SuY,) + I{U2,S2;Y2\Uo) 

-I{Ui,8i;U2,82\Uo), (4) 

i/(Si, S2) < /([/i, Si; Filf/o) + I{Uo, U2, S2; 1^2) 

-/(t/i,Si;[/2,S2|;7o), (5) 

i/(Si, S2) < I{Uo, C/i, Si; Yi) + I{Uo, U2, S2; la) 

- I{Ui, 81; U2, 82\Uo) - /(Si, S2; Uo). (6) 

for some p{uo,ui,U2,x\si, 82)- Here the auxiliary ran- 
dom variable Uq has the cardinality bound \Uo\ < 
min{|A'||Si||S2| +4,|3;i||y2||Si||S2| +4}. 

Remark 1: It is easy to see that Theorem [T| is a special case 
of Theorem |2] In fact, if we define a new random variable 
as Uo ~ {Uo,K), where K denotes the common part of the 
sources (Si,S2), then the inequalities in Theorem |2] for the 
triple (Uo, Ui, U2) simplify and reduce to those in Theorem[T] 
Conversely, the following proposition proves that Theorem |2] 
is a special case of Theorem [T] establishing the equivalence 
of the two. 

Proposition 3: The sufficient condition for reliable trans- 
mission in Theorem [T] is equivalent to the one in Theorem |2] 
Proof: See Appendix A. ■ 

Remark 2: No cardinality bounds are known for the auxil- 
iary random variables Ui and U2- 

Remark 3: Application of Theorem |2] yields the following 
results as special cases: 

a) Marton's inner bound i l27V : Consider the special case 
of independent sources described above, so take Si = 
{Wo,Wi) and S2 {Wo,W2), with Wo, Wi, and 
W2 of entropies Ro, Ri, R2, respectively. By choosing 
{Uo, Ui,U2) to be independent of (T^o, Wi,W2), Theo- 
rem |2] yields Marton's inner bound, which state that the 
capacity region of the DM-BC p{yi,y2\x) contains the 
set of rate triples {Ro, Ri, R2) satisfying 

Ro + Ri<I{Uo,Ui;Yi), 
Ro + R2<I{Uo,U2;Y2), 
Ro + Ri+R2< I{Uo, Ui;Yi)+ /(C/2; Y2\Uo) 

~ I{Ui;U2\Uo), 
Ro + Ri+R2< I{U,;Y,\Uo) + I{Uo, C/2; ^2) 
- I{Ui;U2\Uo), 
2Ro + Ri+R2< I{Uo, Ui;Yi) + I{Uo, C/2; ^2) 
-I{Ui;U2\Uo) 

for some p{uo, ui, W2, x). 

b) The Gray-Wyner source coding problem 4751/ : Consider 
the noiseless DM-BC channel with Yi = {Xo,Xi), 
Y2 = (^0,^2) with links of rate Ro = H{Xo), 
Ri ^ H{Xi), R2 = H{X2). By taking Ui = Xi, 
U2 — X2, and Uo = {Xo,V), under the distribu- 
tion p{v\si, S2)p{xo)p{xi)p{x2), the inequalities in The- 
orem I2] simplify to the following rate region 

Ro + Ri> I{Si,82;V) + H{8i\V), 
Ro + R2> I{Si,82;V) + H{82\V), 
Ro + Ri+R2> /(Si, S2; V) + H{8i\V) + H{82\V), 
2Ro + Ri+R2> 21 {81,82; V) + H{8i\V) 
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+ H{S2\V) 

which includes (and is in fact equivalent to) the Gray- 
Wyner distributed source coding region, which is charac- 
terized by the following set of inequalities: 

i?o >/(5i,^2;1/), 
Ri > H{Si\V), 

i?2 > H{S2\V). 

c) Degraded "source" sets: Consider the case in which 
decoder 1 is interested in reconstructing both source 
sequences (S*", 5*2 ) with vanishing error probability. This 
setup can be captured by considering transmission of a 
pair of sources Si ~ {81,82), S2 = 5*2- By choosing 
Uo = (U,S2), Ui = X, and U2 = const, with 
p{u,x\si,S2) = p{u,x), the conditions stated in the 
Theorem |2] reduce to 

HiS2) < I{U-Y2), 
H{Si,S2) < I{U;Y2)+IiX;Yi\U), (7) 
H{Si,S2) < I{X-Yi). 

Conversely, iflSl shows that if the sources (5*1, 5*2) can be 
reliably transmitted in this setup, then conditions (|7]) with 
"<" replaced by "<" have to be simultaneously satisfied 
for some p{u,x). Thus, we conclude that the sufficient 
condition in Theorem |2] is also necessary for this class of 
problems. It should be remarked that the same condition 
can also be obtained by separately performing source 
and channel coding, first compressing the sources and 
then encoding the resulting sequences using the channel 
coding scheme in lITOI . 

d) More capable broadcast channels: A DM-BC is said to 
be more capable if I{X;Yi) > I{X;Y2) for all p{x) 
(cf. IS). If we apply Theorem |2] to this setting, and we 
choose Uq = {11,82), Ui = X, and U2 = const., with 
p{u, x\si, S2) = p{u, x), then the conditions stated in the 
theorem reduce to (|7]). Conversely, it is shown by Kang 
and Kramer ||9l that if the sources {Si , S2) can be reliably 
transmitted in this setup, then conditions (|7| with "<" 
replaced by "<" have to be simultaneously satisfied for 
some p{u, x). Hence, the sufficient condition in Theorem 
I2] is also necessary for this class of problems, but again 
the same condition can be obtained by performing source 
and channel coding separately. 

IV. Proof of Theorem[2] 

The coding technique used to prove Theorem |2] can be 
outlined as follows. The scheme requires three auxiliary 
random variables {Uo,Ui,U2)- Information about the source 
sequence Si is carried by Uq and Ui, while information about 
the source sequence S2 is carried by Uq and U2- Although 
Uq carries information about both source sequences, it is not 
treated as cloud center in our coding scheme, so an error in 
decoding Uq does not preclude successful decoding of Si or 
S2- Marton's subcode generation technique is used to obtain 
arbitrary correlation among {Uq, 1/1,1/2) ■ 

The following definitions are needed for the remainder of 
the paper 



Definition 4: Let (a;",j/") be a pair of sequences with 
elements drawn from a pair of finite alphabets {X, y). Define 
their joint empirical probability mass function as 

.{x,y\x-,yn ^ \i^--(-^^y^) = (-'y)}\ , for {x,y)eX.y 

n 

Then, the set T/"'(X,r) (in short, t}''^) of jointly e-typical 
n-sequences is defined as: 

T}"\X,Y) = {{x",y'^) ■.\Tv{x,y\x'\y") -p{x,y)\ < ep{x,y) 
for all {x,y) £ X x y}. 

Similarly, the set T/"' (F of e-typical n-sequences that 
are jointly typical with a given sequence is defined as: 

r,(")(y|:r") ^ {y" : G r,(")(x,y)}. 

In the remaining of this section, we first describe the random 
codebook generation and encoding-decoding scheme, then we 
outline the analysis of the probability of error, which is treated 
in detail in Appendix C. 

Random codebook generation: Let e' > 0. Fix a joint 
distribution p{uq,ui,U2\si, S2) and, without loss of gen- 
erality, let p{x\uq,ui,U2, si, S2) be a chosen deterministic 
function x{si, S2,uq,ui,U2)- Compute p{uq), p{ui\si), and 
p{u2\,S2) for the given source distribution p{si,S2). Ran- 
domly and independently generate 2"^° sequences UQ(mo), 
mo G [1 : 2"^"], each according to IlILi J't^o (""oi)- For 
each source sequence s" randomly and independently gen- 
erate 2"^i sequences u"(s",mi), mi G [1 : 2"^^], ac- 
cording to Y\^^iPui\Si{'u-ii\sii). The same procedure, using 
nr=iP'72|S2("2i|s2i), is repeated for generating 2"^^ se- 
quences M^(s^,TO2), 7712 G [1 : 2"^^]. The rates {Rq,Ri,R2) 
are chosen so that the ensemble of generated sequences 
"cover" the set tJ"' (C/q, f/i, C/2|s'i\ s^) for all 
(s;',s5) G T^''\Si,S2). The conditions for ' 'covering" are 
given in the following Lemma. 

Lemma 5: Define the event 

A ={(5r, 52", U^{mQ), C/r(5r, mi), 

U^{S'^,m2) ^ rj"^ for all mo, mi, 7712}. 

Then, there exists (5(e') which tends to zero as e' ^ such 
that P(^) ^ as n ^ cx) if the following inequalities are 
satisfied 

i?o >/(f/o;5i,52) + <5(e'), 
Ri>I{Ui,S2\Si)^8{('), 
R.2> I{U2;Si\S2) + S{e'), 
Ro + Ri> I{Uq; Si,S2) + I{Ui; Uq, S2\Si) + S{e'), 
Rq + R2> I{Uq; Si,S2) + HU2; Uq, Si\S2) + d{e'), 
Ri+R2> I{Ui,Si; U2, S2) - I{Si;S2) + S{e'), 
Rq + Ri+R2> I{Uq; Si,S2) + /(C/i; C/2IC/0, 51,^2) 
+ I{Ui;Uq,S2\Si) 

+ I{U2;UQ,Si\S2) + 5{e'). (8) 

Proof: See Appendix B. ■ 

Encoding: For each (s",S2) G T^*""' (S'l, 6*2), choose a 
triple (mo,mi,m2) G [1 : 2"-"«] x [1 : 2"-"i] x [1 : 2"^=] 
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such that (s", S2 , Uq (mo): (s", mi), ^2 J '^2)) £ 
T^r\Si,S2,Uo,Ui,U2). Choose (mo,mi,m2) 
(1,1,1) if no such triple can be found. Then, 
at time i G {l,...,n}, the encoder transmits 
Xi = x{sii,S2t,uoi{mo),uii{s1,mi),U2i{s^2j'm'2)). The 
sequence so generated is the codeword corresponding to 
the source sequence (s" , s^)- 

Decoding: Let e > e'. Upon observing the se- 
quence y", decoder 1 declares that s" is the transmit- 
ted source sequence if this is the unique sequence such 
that (s?,<(mo),<(s?,mi),2/r) G t}"\SuUo,Ui,Yi) for 
some mo G [1 : 2"-'^o] and mi G [1 : 2"^i]. Otherwise, 
declare an error Similarly, the decoder observing de- 
clares that S2 was sent, if this is the unique sequence such 
that {s^,u^{mo),u^{s^,m2),yE) G t/"' (^2, t/o, C/2, F2) for 
some mo G [1 : 2"^"] and ma G [1 : 2"^^]. 

Error events: The detailed analysis of the error probability 
is given in Appendix C, but it can be outlined as follows. 
If the "covering" conditions in Lemma |5] are satisfied, then 
the probability of error encoding can be made arbitrarily 
small by letting n — > 00. Thus, we focus on the analysis of 
decoding errors. Assume that (mo, mi, ma) = (A/o, -A/i, A/2) 
is selected for the source sequence (s",S2). The error event 
for decoder 1 can be divided into two parts: 

1) There exists a sequence s" ^ such that 
(s?,<(Afo),uns?,mi),yr) G r,(")(5i, i7o, f/i, Fi) 
for some mi G [1 : 2"^^]. The probability of this 
event vanishes as ri 00 if the following inequality 
is satisfied 

HiSi) + Ri < I{UuSi;Uo,Yi) -2S{e). (9) 

2) There exists a sequence s" ^ S'" and an 
ma ^ Afo such that (s" , Wq (toq), w" (s" , mi), y") G 
re*"^(5'i,f7o,;7i,yi) for some mi G [1 : 2"-"i]. The 
probability of this event can be made arbitrarily small 

as n — > 00 if 

HiSi) +Ro + Ri< I{Uo, Ui, Si;Yi) 

+ I{Uo;Ui,Si)-36{e). (10) 

Similarly, the probability of error for decoder 2 can be made 
arbitrarily small as n tends to infinity if 

HiS2) + R2<I{U2,S2;Uo,Y2)-2Sie), (11) 
H{S2) + Ro + R2< I{Uo, U2, S2;Y2) 

+ I{Uo;U2,S2)-3S{e). (12) 

Fourier-Motzkin elimination: In summary, by combining the 
inequalities in Lemma |5] (|9]l, ( fTOl i. ( fTTl i. and (fT2T i. and by 
letting e',e 0, we conclude that the sources (6*1, S'2) can 
be reliably transmitted over the DM-BC p{yi,y2\x) if there 
exists a rate tuple {Rq, Ri, R2) satisfying 

Ro > I{Uo; 5*1, 5*2) = VI, 
Ri > I{Ui;S2\Si)^v2, 

R2> I{U2;Sl\S2)=V3, 

Ro + Ri> I{Uo;Si,S2)+I{Ui;Uo,S2\Si)^v4, 
Ro + R2> I{Uo; Si,S2) + I{U2; Uo,Si\S2) = v^, 



Ri+R2> I{Ui,Si;U2,S2) - I {Si; S2) = ve, 
Ro + Ri +R2> I{Uo; 81,82) + I{Uv, f/alf/o, 5*1, 5*2) 
+ I{Ui;Uo,82\8i) 
+ I{U2;Uo,8i\82) = V7. 
H{8i)+Ri <I{Ui,8i;Uo,Yi)=v8, 
H{8i)+Rq + Ri < I{Uo,Ui,8i;Yi) + I{Uo;Ui,8i) ^ V9, 

H{82} + R2 < I{U2,82;Uo,Y2) ^ vio, 
H{82)+Ro + R2<I{Uo,U2,82;Y2)+I{Uo;U2,82)=vi-, 

(13) 

for some p{uo,ui,U2, x\si, 82)- The final step in our deriva- 
tion consists of eliminating the auxiliary variables Ro, Ri, 
R2 from the above set of inequalities. This routine task can 
be performed by application of the standard Fourier-Motzkin 
elimination algorithm ifTTl 1271 . Unfortunately, brute force 
application of this algorithm to eliminate Rq, Ri, R2 (in 
this order) results in 28 inequalities involving H{Si), H{S2), 
and H{Si, S2), most of which are redundant because implied 
by other inequalities. Also, to find a minimal set of non- 
redundant inequalities, one has to verify linear information 
inequalities involving several random variables. Therefore, 
although conceptually simple, application of the Fourier- 
Motzkin elimination algorithm can be tedious. 

We would like to illustrate a technique that allow us to 
get around these difficulties and to efficiently perform the 
elimination using a computer program. The key idea is to treat 
the right hand sides of (flJl i as auxiliary variables. Denote them 
by vi, . . . ,vii. Next, observe that 

VI > 0, 

V2 > 0, 

V3 > 0, 
V1+V2 < V4, 

Vl + V3 < v^, 

V2+VZ < ve, (14) 

V4+VQ < V2+ vt, 

+ < Vl + vt, 
V5 +ve < V3 + vt, 

V8 < Vg, 

vio < vii. 

The above information inequalities can be checked directly 
using the chain rule and the nonnegativity of mutual infor- 
mation or, alternatively, they can be verified by the software 
ITIP l25l . 

Combining (fT3] l and ( fT4] l we obtain a system 
of linear inequalities involving the variables 
{H{Si),H{S2),Ro,Ri,R2,vi,V2,...,vu). Next, we 
eliminate the auxiliary variables _Ro, Ri and R2 from this 
system of linear inequalities. Since (l3[ and (fT4l i have 
constant coefficients, the Fourier-Motzkin elimination can 
now be performed by a computer program, e.g., by the 
software PORTA |[T|. The algorithm results in the following 
inequahties involving H{Si) and H{S2): 

H{Si) < Wg — V4, 
H{Si) < W8 - V2, 

H{S2) < vii - U5, 
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H{S2) < Vio - W3, 

H{Si) + H{S2) < vg + Via " vr, 
H{Si) + H{S2) < vs + vii - vt, 
H{Si)+H{S2) < Vs+vio-vq, 
H{Si)+H{S2) <vg+vn-V7-vi. (15) 

To complete the proof, we argue that three of the above 
inequalities can be discarded because inactive. We proceed 
as follows. Define three new auxiliary random variables Ui = 
{Ui,W), i G {0,1,2}, where W is chosen independent of 
everything else, and replace Ui with Ui in (T3[ . Substituting 
the new values of vi, . . . ,vii, and performing some manipu- 
lations, we can rewrite dTSI l as follows 

H{Si) < I{Uo,Ui,Si;Yi) - I{Uo,Ui;S2\Si), 
H{Si) < I{UuSi;Uo,Yi) 

-I{Ui;S2\Si)+H{W), (16) 
H{S2) < I{U„, U2, S2;Y2) - I{U„, U2; ^il^a), 
H{S2) < I{U2,S2;Uo,Y2) 

-I{U2;Si\S2)+H{W), (17) 
H{Si,S2)<I{U„,Ui,Si;Yi)+I{U2,S2;Y2\Uo), 

-I{Ui,Si;U2,S2\Uo), 
H{Si,S2) < I{Ui,Si;Yi\Uo) + I{U„, C/2, ^2; Y2) 

-I{Ui,Si;U2,S2\Uo), 
H{Si,S2) < /([/2, ^2; t/o, Y2) + I{Ui, Si; Un, Yi) 

~I{UuSi;U2,S2)+H{W), (18) 
H{Si,S2) < /(t/o, C/i, Si;Yi) + I{Ua, C/2, ^2; Y2) 

- I{UuSi;U2,S2\Uo) ^ I{Si,S2;Uo). 

By letting H{W) 00, we can make ( fT6] l. (Tj) and (fTSl l triv- 
ial, and thus inactive, while leaving the remaining inequalities 
unchanged. In summary, the five inequalities remaining from 
the Fourier-Motzkin elimination are those appearing in the 
statement of Theorem |2] This proves the desired sufficiency. 

V. Discussion 

The best known inner bound for the problem of broadcasting 
correlated sources over a general DM-BC is due to Han 
and Costa with the associated coding scheme that cleverly 
combines a melange of coding techniques, including joint 
typicality encoding and decoding, random hashing, super- 
position coding, and the use of the common part between 
two random variables. To investigate which techniques are 
crucial, this paper presents a "minimalistic" coding scheme 
in which we remove from the coding scheme of Han and 
Costa all unnecessary components without affecting the overall 
performance. Our proposed coding scheme does not require 
random hashing and superposition coding, and it does not 
involve the common part of two random variables. This 
highlights how source encoding can be performed by simply 
jointly "covering" the set of typical sources using auxiliary 
correlated random variables. An interesting implication of our 
result, albeit no more than theoretically amusing, is that the 
capacity of the degraded DM-BC can be achieved without 
employing Cover's superposition coding 



We would like to conclude the paper by showing that the 
sufficient condition stated in Theorem |2] can also be proved 
by means of a more elaborate coding scheme which uses 
superposition coding in addition to joint typicality encoding 
and decoding. This alternative scheme differs from the one 
described in Section IV in the way the random codebook 
is generated, as the auxiliary random variable Uq serves as 
a cloud center for generating the auxiliary random variables 
Ui and U2- Appendix D presents a description of the code 
construction and a sketch of the analysis of the associated 
probability of error. 



Appendix A 
Proof of Proposition^ 

Suppose that the source (5*1, 52) satisfies the sufficient 
condition stated in Theorem |2] so that the inequalities in 
Theorem |2] are satisfied for some triple {Uq,Ui,U2)- The goal 
is to show that {Si, S2) also satisfy the inequalities in Theorem 
[T]for the same triple of auxiliary random variables. Expanding 
the left hand side of (|4]i, we obtain that 

H{Si,S2) 

< I{U„, Ui,Si;Yi) + I{U2,S2; Y2\Uo) 
-I{Ui,Si;U2,S2\Uo), 

= H{Yi) - H{Yi\Uq, Ui, Si) - H{S2, U2\Y2, Uq) 
+ H{S2,U2\Uo,Ui,Si) 

< H{Yi) - H{Yi\K, Uo,Ui,Si)^ H{S2, U2\Y2,K, Uq) 
+ H{S2,U2\K,U„,Ui,Si), 

= I{K, Uo, Ui, Si;Yi) + I{U2, S2;Y2\K, Uo) 
-I{Ui,Si;U2,S2\K,Uo), 

where the second inequality follows from the fact that condi- 
tioning reduces the entropy and that K = f{Si) ~ g{S2) is a 
deterministic function of the sources (5i,iS'2). Thus, {Si,S2) 
satisfy ([T]i. Proceeding in a similar way, it is immediate to 
show that {Si, S2) satisfy (|2]i. Finally, we expand the left hand 
side of ^ and obtain that 

H{Si,S2) 

< I{U„, Ui,Si;Yi) + J(f/o, U2, S2;Y2) 
-I{Ui,Si;U2,S2\Uo)- I{Si ,S2;Uo). 

= H{Yi) - H{Yi\U„, Ui,Si)- H{Y2) - H{Y2\Uo, U2, S2) 

- H{Ui\Si, Uo) + H{Ui,Si\Uo, U2, S2) - H{Si) 

< H{Yi) - H{Yi\K, Uo, Ui,Si) - H{Y2) 

- H{Y2\K,Uo,U2,S2) - H{Ui\Si,K,Uo) 
+ H{Ui,Si\K, Uo, U2, S2) - H{Si) 

= I{K, Uo, Ui,Si;Yi) + I{K, Uo, U2, 52:^2) 

- I{Ui,Si; U2, S2\K, Uo) - I{Si,S2;K, Uo), 

so the sources {Si,S2) satisfy ([3]i. Thus, we conclude that 
{Si, S2) also satisfy the sufficient condition stated in Theorem 

m 
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Appendix B 
Proof of Lemma|5] 



The technique used in the proof is similar to the one in Q. 
For each {s'^,s'^) G rj"'(S'i, S'2), let 

= {(mo, mi, 7712) e [1 : x [1 : 2"-^i] x [1 : 2"^^] : 

(s^ 4, C/^'(mo), C/^ (s^ mi), C/2"(4, m^)) G tJ")}. 
Then, 

P(^)<P(K,4)^rj")) 



By the law of large number the first term in the sum tends to 
as n ^ 00. To bound the second term, observe that 

Using indicator random variables, we express \A{si, Sj)] as 



mo ,mi ,m2 



where 



A„o,mi,™. = {K,4\C^o"('™o),C/r(sr,mi),C/^'(4,m2)) 

Then, taking expectations and using the fact that codewords 
are independently generated, 

E[|^(s-^ , S2 ) I] = ^ ^ P(-^?Tio,nii,Tn2) 

mo ,mi ,7712 

= 2"(^°+^i+-"^)p(A.i,i) (20) 

Similarly, we have 

tmq ,mi ,m2 

~l" ^ ^ ^ ^ P(-^mo, 7771, 7712 ? >^m^, mi, m2 

777,0,7711 ,7?7. 2 771^7^7710 

TTIQ ,7711 !7712 ? '^771Q ,777^ ,7772 

7770 ,7771 ,"72 m,'^^m\ 

"t" ^ ^ ^ ^ P(-^777o, 7771, 7772 , '^7710 , 7771,7772 

7770,7771,7712 

^ ^ ^ ^ P(-^777o, 7771, 7712 , >^771^, 777^, 7772 

7770 , 7711,7712 mo/771o 
771^ 7^7711 

+ ^ ^ ^ ^ P(-^7770, 7711, 7772 , ^77lf|, 7711, 771^ 

7770,7711,1712 771q/771o 
7712 7^7112 

H~ ^ ^ ^ ^ P(-^777o, 7711, 777 2 , ^7710,771^,7712 

7770,7171,7772 m{^mi 
7712 7^7712 



+ ^ ^ ^ ^ P(-^777o ,7771 ,777 2 , -^77lJ, ,77l'j ,711^ ) • 

7770,7711,7712 mf, 7^7710 

7711 7^7711 

7712 7^7712 

Since the codewords are independently generated, we can re- 
write the above equaUty as follows 

E|^P < 2"(^"+^i+^'^'P(^i.i,i) 

+ 2"(2i?,«+i?,i+ii2)p(_4^_^^^^^2,i,i) 

+ 2"(^°+2«i+^^)P(A,l,l,^l,2,l) 

+ 2"(^"+^i+2^^)p(A,i,i, ^1,1,2) 

+ 2"(2«o+2Hi+i?2)p(^^^^_^^^^_^^^) 

+ 2"(2«o+i?i+2fl2)p(^^^^^^^_4^^^^^) 

+ 2"(««+2^^^+2«^)P(^l.l,l,A,2.2) 
+ 2"(2flo+2Hi+2i?.)p(^^^^^^^^^^^^^)_ 

It is easily seen that P(^i.i,i, ^42.2,2) — P(-4i^i.i)^. It follows 
that 

Var(l^l) < 2"(^«+^i+^^^)p(y^ia,i) 

+ 2"(2^«+^i+^-^)p(A,ia, 

+ 2"(^«+2«i+^-^)p(A,ia,A,2,i) 

+ 2"(^»+^i+2«^)P(^l,l,l,A,l,2) 
+ 2"(2«o+2i^i+fl2)p(^^^^_^^^^^2^^) 

+ 2"(2«o+i?l+2H2)p(^^^^_^^^2^^^2) 

+ 2"(^«+2«i+2^^=)p(^ia,i,A,2,2). (21) 
By the joint typicality lemma H), we have 

P{A2 1 1, A 1 1) < 2""[^(^«''^i"^^'^i''^2)+^('^i''^'''^2l'^i)l 

. 2-«[-f(t'o;Si,S2)+/((72;C/o,C/i,Si|S2)]-<5(c')] 

P{Ai 2 1, A 1 1) < 2-"[^(^i^'^^'^"'^^l'^i)+^(^i'^«"^=l'^i)l 

. 2-"[-f(C/o;Si,S2)+/(C/2;;7o,C/i,Si|S2)]-5(e')] 
p(^^ 1 2, A 1 1) < 2""[^(^^''^i''^°'^il'^^)+^(^i'^'''^2l'^i)l 

. 2-"[-f(t'o;Si,S2)+/((72;C/o,C/i,Si|S2)]-<5(c')] 

P(A 2 1, A 1 1) < 2~"[^('^°'^i''^^'^i'^=)+^('^i''^^'^^l'^i)l 

, 2-«[-f(t'i;%.S2|Si)+7(;7o;Si,S2)] 
. 2-n[I{U2;Uo.UuSi\S2)]-'S{e')] 

P{A2 1 2, A 1 1) < 2""[^(^°''^i'^^'^i''^2)+^(^^''^i'^il'^^)l 

. 2-n[I(Ui:Uo,S2\Si)+I{Uo;SuS2}] 
. 2-«[-f(t'2;;7o,C/i,Si|S2)]-5(e')] 

P(A 2 2, A 1 1) < 2""[^(^i'^°'^^l'^i)+^(^^'^°'^i'^il'^^)l 
. 2-"[-f(C/i;C^o.S2|Si)+7(C/o;Si,S2)] 

, 2-n[I{U2-Mo.UuSi\S2)]-S{e')] 

P(^i 1 1) > 2""[^(^o^'^i''^2'+-^(^i'^0''^"l'^i' 

, 2-n[+7(C/2;(7o,C/i,Si|S2)]+5(£')] 

Substituting the above inequalities into (|20] | and (ISTT l. and 
making use of (fT9] l, we obtain that P(|^(s", Sj)! = 0) 
as 71 ^ cx) if conditions (O are simultaneously satisfied. This 
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completes the proof of the lemma. 

Appendix C 
Analysis of the Probability of Error 

Assume that [mQ, mi, 1712) ~ {Mq, Mi, M2) for the trans- 
mitted source sequences (s" , ). To study the error probabil- 
ity for decoder 1, define the events 

£1 ^ {iSl\U^iMo),X^iSl\Mo),Yn i r,(")}, 

£2 " c/o"(Mo), u^iTi, mi), n") e r,(") 

for some ^ S'l and toi G [1 : 2"-"^]}, 

for some ~s1 ^ S'l\ mo Afo, and mi G [1 : 2"-"!]}. 

The average probability of decoding error at decoder 1 is 
bounded by 

p(5i" + 5n < ?{£i) + p(f2) + p(f3). 

By the law of large numbers, P(£i) — > as n — > 00. Next, 
consider the second term. By the union bound, we have 

P(^2) 

= Y,v{s\)P{{ri, USiMo), U^{S"i, mi),Yn G T}"^ 

for some ^ S'i, and mi G [1 : 2"-"^] | = s^} 
<E?'(^i) E E ^i(^~i'^o(Mo),C/i"(s7,mi), 

^1 ^1 T^sf mi = l 

yi") G T,(") I SI ^si}. 

Conditioned on S*" = s", for all s" ^ s" and for all 

mi G [1 : 2"-Ri], we have that {UJ^{Ma),U'1(s1,mi) 
,yi") -pK(s?,mi)|S?) pK(Mo),yr|s?). Thus, 



P(^2) 

< E^'(^i) 



■pK(s7,mi)|s-) 

= E E E ?'W(Mo),yl>?) 

•pK(s7,mi)|s7)p(s5') 

E EE v{<{M,\y-\s\) 

•pK(s7,mi)|?/)pK) 

2"-"i 

E E P("o(^^o),2/nPK(Sr,mi)|5?) 



(.1) Tni=l 



< 2""(-f^('Si,'^o,;7l,yi)+«(e))+"fli-n(-fi"(t'o,yi)+-H(C/l|Si)-25(e)) 



Collecting the entropy terms at the exponent, we have 

H{Si,V^,Ui,Yi) - H{U^,Yi) - H{\Ji\Si) 
= H{Si) + H{Ui\Si) + i/([/o, yi|[/i, 5i) 

-H{lJ^,Yi)-B{Ui\Si) 
= H{Si)-I{Ui,Si;Uo,Yi). 

Thus P(f2) ^ as n ^ 00 if 

H{Si) + Ri< I{Ui, Si;Uo,Yi)- 2J(e). (22) 

Finally, consider the third term. By the union bound, we have 

P(^3) 

= Y.p{s-i)P{{S^^, C/o"(mo), U-{~s-i, mi), y,") G T^") 

^1 

for some s" ^ 5", mo 7^ A/o, and mi|5" = s"} 

<E^^(*i) E E E ^{(^~i'^o"("^o),c/r(s?,mi), 

s^'^^^sj mo = l mi = l 

yi") G r,(")|5r - s?}. 

Conditioned on 5^' = s^, for all ^ s^, mo ^ Mq, 
and for all mi, we have that {U^{mo),Ul'{s'l,mi),Y{') ~ 
p(MjJ(mo))p(u5' (s^* , mi) I s'l)p{y'l | ). Thus, 



P(^3) 

<Ep(^i) 



E E E^^K("^o)) 

■pK(s7,mi)|s7)p(yr|s?) 

2"«C) 2"-"i 

E EE E^^«(-o)) 

•pK(s7,mi)|s7)p(y^'|s^)pK) 

^ E E E E pK(-o)) 

•p«(s7,mi)|?/)p(yr|s?)pK) 

= E E E^^«(™o)) 

.p«(.7,mi)|.7)p(2/r) 

(s5',u5,M5','y;;')GTi"' 

^ 2-"(^^(S'i.'^o,C/i,n)+«(e))2"(^o+-Ri) 
. 2-«(ff(C^o)+-H"(!7i|Si)+H(yi)-35(e)) 

Collecting the entropy terms at the exponent, we have 

H{Si,Uo,Ui,Yi) - H{Ua) ~ H{Ui\Si) - H{Yi) 
= H{Si) + H{Ui\Si) + H{Uo\Ui, Si) + H{Yi\Si, Uo, Ui) 

- H{Un) ~ H{Ui\Si) - H{Yi) 
= H{Si)-I{Uo,Ui,Si;Yi)- I{Uo; Ui,Si). 
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Thus P{£3) ^ as 77 ^ cxo if 

HiSi) + Ro + Ri <I{Uo,Ui,Si;Yi)+ 

+ I{Uo;Ui,Si)-3S{e). (23) 

In summary, the probabiHty of error for decoder 1 can be 
made arbitrarily small by letting 7i ^ cxa if (|22] | and ( |23] | 
hold. Finally, the study of the error probability for decoder 2 
follows from a similar argument. 

Appendix D 
An Alternative Coding Scheme Using 
Superposition Coding 

We describe here the construction of coding scheme based 
on superposition coding which yields the same sufficient 
condition as the one stated in Theorem |2] 

Random codebook generation: Let e' > 0. Fix a 
joint distribution p{uo,ui,U2\si, S2) and, without loss 
of generality, let p(a:|uo, "i, "2, si, S2) be a chosen de- 
terministic function x{si, S2,uo,ui,U2)- Compute p{uo), 
p{ui\uo,si) and p{u2\uo,S2) for the given source dis- 
tribution p{si,S2)- Randomly and independently generate 
2ri-Ro sequences Uq(too)i ™o G [1 : 2"-'^"], each ac- 
cording to n"=i Pc/o ('^oi)- For ^^ch source sequence s" 
and UQ (r77o), randomly and independently generate 2"^^ se- 
quences u"(s", mo, ?77i), 7771 G [1 : 2"^i], each according to 
117=1 PUi\SuUoi't^i,i\sii,uo,i{mo)). Similarly, for each source 
sequence ™d itQ (7779), randomly and independently gener- 
ate 2"^^ sequences U2{s2,mo,m2), 7712 £ [1 : 2"^^], each 
according to nr=iPc/2|S2,c/o("2,i|s2,i,uo,i(TOo))- The rates 
{Ro,Ri,R2) are chosen so that the ensemble of generated 
sequences "cover" the set T}r\UQ,Ui,U2\si, S2) for all 
e Tj"^(5i, S2). Define the event 

A U^\mo), U^' {Sl\mo, mi), 

112(3 2, mo, 1712)) ^Tj"-' for all mo,mi,m2} 

It can be shown using techniques similar to those used in the 
proof of the Lemma |5] that P(^) ^ as 77 —> 00 if 

Ro > I{Uo;SuS2)+S{e'), 
Ro + Ri> I{Uo;Si,S2)+I{S2;Ui\SuUo) + S{e'), 
Ro + R2> HUo; Si,S2) + /(Si; U2\S2, Uo) + S{e'), 
Ro + Ri + R2> HUo; Si, S2) + I{S2; Ui\Si, Uo) 
+IiSi,Ui;U2\S2,Uo)+5ie') 

(24) 

where 5{e') — > as e' ^ 0. 

Encoding: For each source sequence (sijSj), choose a 
triple (mo,mi,m2) £ [1 : 2"^«] x [1 : 2"^i] x [1 : 2""-'] such 
that (s^,s^,M^(mo),Mi(s'i\mo,mi),it5(s5,mo,m2)) £ 
Tj")(S'i,S'2,C/o,f/i,C72). If there is no such 
triple, choose {mo, mi, 1112) = (1,1,1). Then 
at time i G [1 : 77], the encoder transmits 
Xi = x{sii,S2i,UQi{mo),Uii{s1,mQ,mi),U2iis^,mo,m2)). 

Decoding: Let e > 0. Decoder 1 declares to be the 
estimate of the source s" if it is the unique sequence such that 

(s'i\7z^(mo),<(si",mo,mi),2/lO G %^'''\Si,Uo,Ui,Yi) 
for some (mo, mi) e [1 : 2"^«] x [1 : 2"-"i]. 
Similarly, decoder 2 declares to be the estimate of 



the source itj if it is the unique sequence such that 

{s^i,u^{mo),u^{rs^,mo,m2),yh') G T,'"^ (^2, t/o, f/2, ^2) for 
some (mo,m2) S [1 : 2"-^«] x [1 : 2"^^]. 

Error events: Assume that {mo,mi,m2) = {Mo,Mi,M2) 
is selected for the source sequence (s",S2)- The error event 
for decoder 1 can be divided into two parts: 

1) There exists a sequence s" ^ 5" such 
that {s1,u^{Mo),u^{s^,Mo,mi),v^) £ 
T}''\Si,Uo,Ui,Yi) for some mi G [1 : 2"-"i]. 
The probability of this event vanishes as ti ^ cx) if the 
following inequality is satisfied 

H{Si) + Ri< I{Ui, Si;Yi\Uo) + I{Si; Uo) - 5{e)- 

(25) 

2) There exists a sequence ^ Si and an mo ^ 
Mo such that {s1,u^{mo),u^{sl,mo,mi),yD G 
%^'^\Si,Uo,Ui,Yi) for some mi G [1 : 2"-«i]. The 
probability of this event can be made arbitrarily small 
as 77 —> 00 if 

H{Si) +Ro + Ri< I{Uo, Ui, Si;Yi) + 

+ I{Uo;Si)~d{e). (26) 

Similarly, the probability of error for decoder 2 can be made 
arbitrarily small as 77 ^ 00 if 

H{S2) + R2 < I{U2,S2;Y2\Uo) + I{S2;Uo)-Sie), 

H{S2) + R0 + R2 < I{Uo, U2, S2;Y2) + I{Uo; S2) - S{e). 

(27) 

The rest of the proof follows by letting e', e ^ 0, then 71, ^ 00, 
and finally by eUminating (Ro, i?i, i?2) from (|24li, (|25]l, ( |26] l. 
and dZTl l using the Fourier-Motzkin elimination algorithm. 
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